Many computer systems today use cache memories to improve the speed of access to more frequently used data and instructions. A small cache memory may be integrated on a microprocessor chip itself, thus, greatly improving the speed of access by eliminating the need to go outside the microprocessor chip to access data or instructions from an external memory.
During a normal data load accessing routine, the microprocessor will first look to an on-chip cache memory to see if the desired data or instructions are resident there. If they are not, the microprocessor will then look to an off-chip memory. On-chip memory, or cache memory, is smaller than main memory. Multiple main memory locations may be mapped into the cache memory. The main memory locations, or addresses, which represent the most frequently used data and instructions get mapped into the cache memory. Cache memory entries must contain not only data, but also enough information ("tag address and status" bits) about the address associated with the data in order to effectively communicate which external, or main memory, addresses have been mapped into the cache memory. To improve the percentage of finding the memory address in the cache (the cache "hit ratio") it is desirable for cache memories to be set associative, e.g., a particular location in memory may be stored in multiple ways in the cache memory.
Most previous cache designs, because of their low frequency, can afford a relatively large cache, e.g. a cache which contains both integer data and larger floating point data. However, as microprocessor frequencies and instruction issue width increase, the performance of on-chip cache system becomes more and more important. In cache design, low latency and high capacity requirements are incompatible. For example, a cache with a low latency access usually means the cache has a small capacity. Conversely, a large cache means the cache has a long access latency.
For the reasons stated above, and for other reasons stated below which will become apparent to those skilled in the art upon reading and understanding the present specification, it is desirable to develop improved performance for on-chip cache memory.